Methods and Apparatus for Handheld Tool

ABSTRACT

In exemplary implementations of this invention, a computer-assisted, handheld machining tool allows even an inexperienced user to carve a complex 3D object, while maintaining artistic freedom to modify the sculpture from an initial CAD design. The tool prevents the user from unintentionally removing material from a volume defined by the CAD design. It does so by slowing or halting spindle rotation as the bit approaches or penetrates the protected volume. The user can override this protective feature. The tool may operate in at least three interaction modes: (i) a static mode in which a static CAD model is used, where the computer assists by preventing the user from damaging the static model; (ii) a dynamic mode where the computer dynamically modifies the CAD model during the sculpting process; and (iii) an autonomous mode where the computer can operate independently of the user, for tasks such as semi-automatic texture rendering.

RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.13/784,674 filed Mar. 4, 2013, which claims the benefit of U.S.Provisional Application Ser. No. 61/605,941 filed Mar. 2, 2012.

FIELD OF THE TECHNOLOGY

The present invention relates generally to subtractive manufacturing andadditive manufacturing.

SUMMARY

In exemplary implementations of this invention, a computer-assisted,handheld machining tool allows even an inexperienced user to carvecomplex 3D objects, while maintaining artistic freedom to modify thesculpture from an initial CAD design. The tool prevents the user fromunintentionally removing material from a volume defined by a CAD modelof the object being carved. If the user tries to move the bit of thetool into that protected volume, the rotational speed of the bit slowsas the bit approaches a surface of that volume, and halts completely ifthe bit penetrates a surface of that volume.

However, the user can intentionally this protective feature and removematerial even from the protected volume. If the user provides input tooverride the protective feature, then the rotation of the bit does notslow or stop as the bit approaches and penetrates the volume defined bythe CAD model.

The handheld machining tool may operate in at least three interactionmodes: (i) a static mode in which a static CAD model is used, where thecomputer assists by preventing the user from damaging the static model;(ii) a dynamic mode where the computer dynamically modifies the CADmodel during the sculpting process; and (iii) an autonomous mode wherethe computer can operate independently of the user, for tasks such assemi-automatic texture rendering.

In the static mode, the CAD model does not change. However, the userretains artistic freedom in any mode of operation of the tool, includingthe static mode. For example, the user can choose how closely andsmoothly the resulting sculpture adheres to the surface of the protectedvolume, even while the bit remains outside that surface. Or, forexample, the user can select features from multiple CAD models to createa hybrid shape that exists only in the resulting physical sculpture, notin any CAD model.

In the dynamic mode, artistic freedom is further enhanced. The computermodifies the CAD model in response to movements of the tool relative tothe object being sculpted and to other user input, so that the user cancause the CAD model to change during the sculpting process.

The user may interact with the handheld tool in at least three differentways during dynamic mode operation: (i) direct shape deformation (DDS);(ii) volume occupancy optimization (VOO), and (iii) data-driven shapeexploration (DDSE). In all three of these cases (DDS, VOO, and DDSE), ifthe user moves the bit into the volume defined by the virtual mode, thecomputer responds by dynamically modifying the CAD model. In each case,as the user removes material from the protected volume, the CAD modelchanges. The user can provide input indicating a selection of the typeof dynamic mode operation. Alternately, the tool may have only one typeof dynamic mode operation.

In direct shape deformation (DDS), the computer deforms the mesh of theCAD model in response to the bit penetrating the volume defined by thatmodel. For example, if the bit penetrates the neck of a virtual giraffemodel from the left, the computer can smoothly deform the mesh, so thatthe neck of the virtual giraffe model bends to the right.

In VOO, the computer optimizes the CAD model to best fit the remainingmaterial of the object being sculpted.

In DDSE, the computer searches a database of many different CAD models,in order to select a CAD model that best fits the remaining material ofthe object being sculpted. The computer then employs volume occupancyoptimization to fine-tune the CAD model to fit the remaining material.

In autonomous mode, the tool can operate independently of the user forsmall scale movements, for example to create rough or smooth surfaces.

Thus, the handheld rotary carving tool allows the user personalize andmodify an underlying 3D model while fabricating the 3D object.

The description of the present invention in the Summary and Abstractsections hereof is just a summary. It is intended only to give a generalintroduction to some illustrative implementations of this invention. Itdoes not describe all of the details of this invention. This inventionmay be implemented in many other ways.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a fabrication system including, among other things, ahandheld machining tool, a computer, a monitor, and a magnetic motiontracking system.

FIG. 2A is a side view of a handheld machining tool.

FIG. 2B is a cutaway view of the handheld machining tool, showing someinternal parts of the tool.

FIGS. 3A and 3B show examples of how the handle of the handheldmachining tool can move relative to the spindle of the tool. In FIG. 3A,the movement is lateral. In FIG. 3B, the movement is up and down in acarving plane.

FIG. 4 is a high level diagram showing data flow in software for thehandheld machining tool.

FIG. 5 is a high level diagram of control software for the tool.

FIG. 6A is a visualization of a CAD model of a tiger.

FIG. 6B is a photograph of a sculpture of the tiger, which wasfabricated using a prototype of the handheld machining tool.

FIG. 7 shows a fabricated object with a hybrid 3D shape that matchesparts of three different meshes. The hybrid 3D shape does not existvirtually; it exists only in the fabricated object.

FIG. 8 is a photograph of a sculpture, produced by a human useroverriding an original CAD model of a tiger.

FIGS. 9A, 9B, 9C illustrate model deformation when the human user of thehandheld machining tool overrides an original CAD model. FIG. 9A shows asculpture produced in accordance with the original CAD model, without auser override. FIG. 9B shows model deformation from the left. FIG. 9Cshows model deformation from multiple directions.

FIG. 10 is a photograph showing, among other things, regions machinedusing autonomous operation of a handheld machining tool.

The above Figures illustrate some illustrative implementations of thisinvention, or provide information that relates to those implementations.However, this invention may be implemented in many other ways. The aboveFigures do not show all of the details of this invention.

DETAILED DESCRIPTION

In exemplary implementations of this invention, a computer-assistedhandheld machining tool is tracked and controlled with reference to aCAD model. The tool allows unskilled makers to produce complex carvingtasks, and to personalize and modify digital 3D models while physicallycarving. The control software offers guidance according to static CADmodels or dynamic ones, which may be altered directly or parametrically.In addition, the machining tool is also able to semi-autonomously moveand carve. The machining tool is configured to allow a human artisan (a)to violate the virtual guidelines, and (b) to impart personalinterpretations into the sculpture. This creates synergetic cooperationbetween human and machine that ensures accuracy in recreation of themodel while preserving the expressiveness of manual carving.

For example, in a prototype of this invention, the tool can be used tocarve a sculpture of an alien humanoid from balsa foam, relying on acomputational 3D model. The human user can make decisions during themachining process, resulting in a unique, personalized sculpture. Or,for example, this prototype can be used even by a person with no priorsculpting training to carve a gargoyle sculpture with a wingspan of 280mm based on a CAD model.

In this prototype, a handheld machining tool is monitored by a computerand a magnetic tracker, while a user freely moves the machining tool.Sensors and a computer track the location of the bit in relation to aCAD model, using this information to control the speed and alignment ofthe shaft. The user can choose to ignore the computer, thus transformingthe tool from a computational-aided device to a purely manual one.Sensors allow the handheld machining tool to also serve as an inputdevice. For example, data indicative of the position of the bit may begathered by the sensors. This data may be used by the computer to, amongother things: (i) determine how to react to user's actions, or (ii)guide semi-autonomous machining by the tool (e.g., in which the toolmakes small-scale carving motions independently of the user).

In exemplary implementations, the handheld machining tool has at leastthree operational modes: (i) a static mode (in which the CAD model isrigid), (ii) a dynamic mode (in which a computational CAD model respondsto physical movement of the machining tool relative to the object beingcarved and to other user actions), and (ii) a semi-autonomous mode oftool operation.

This invention may be implemented as a computer-assisted sculptingsystem comprises a handheld machining tool, a magnetic motion trackingsystem (MMTS), a control computer, and software distributed over thecomputer and the tool. The machining tool is held with a single hand,while the user is free to move it in 3D, limited only by the length ofpower cables and the MMTS.

FIG. 1 shows an example of such a system. A handheld machining tool 101can be held by a user in one hand, in order to carve an object 103. Theobject being sculpted 103 may rest on a table 105 or pedestal. The MMTSmay be used for tracking position of the handheld machining toolrelative to the object being sculpted. The MMTS may comprise: (i) amagnetic field generator 107 located in a fixed position relative to(and close to) the object being sculpted, (ii) a magnetic sensor 207located on the machining tool 101 and (iii) and a processor 109configured to control the MMTS. A computer 111 may be used to controlthe system, including controlling a graphical user interface displayedon an electronic display screen 115 (e.g., a computer monitor or othercomputer screen).

In the example shown in FIG. 1, the system integrates trackingcapabilities, digital drivers for the DC and servo motors, and an optionfor the user to override a CAD model. Preferably, motors for themachining tool are located as far as possible from the MMTS sensor onthe machining tool, in order to minimize the potential for magneticnoises.

In a prototype of this invention, the handheld tool includes a rotarycutting mechanism (including a spindle) built on top of a long shaft andconnected to a 12V DC motor (Micro-Drives® M2232U12VCS with up to 10,000RPM with no load, and up to 5.2 mNm torque). The 6.35 mm round bit ismade from steel. A 3D bearing mechanism is located in the interior ofthe tool, adjacent to the titanium shaft. The bearing enables 3 degreesof freedom (DOF) movement at an approximate spherical volume of 20 mm.

In this prototype, three servomotors in the handheld tool determine theshaft's position. These servos are aligned perpendicular to the shaftand near the spindle motor.

In this prototype, the three servos comprise MKS 6125 mini servos whichare strong (up to 5.8 kg-cm for 6V), small, and lightweight (25.26 g).An electronic circuit on the PCB (printed circuit board) of the handheldmachining tool communicates with the main computer 111 via Bluetooth tocontrol both the shaft movement and the spindle speed. The PCB includesan ATmega328 microprocessor and a MC33926 motor driver, and is poweredwith 5V and 12V power signals.

In this prototype, a force-sensing resistor (FSR) pressure sensor islocated on the handle of the handheld machining tool. One or moreprocessors output control signals to cause the DC motor speed (Sp, when1 is the maximal value) to be a linear factor of the pressure (Pr, when1 is maximal value) and the risk to the model (Rs, 1 is maximal risk):

Sp=1−Rs(1−Pr)  (Eq. 1)

In this prototype, two LEDs are located on the tool, to provide the userwith visual feedback. The first LED blinks with correlation to thepressure detected by the FSR sensor (constant light represents maximalpressure, and no light represent no pressure). The frequency of thesecond LED corresponds to the distance between the bit and the surfaceof the model (when the bit touches the model's surface the light isconstant). In addition, the operation frequency of the DC motor (PWM),controlled by the motor-driver, is changing from ultrasonic to anaudible range (around 2 KHz) to give the user an alarm when the bit isapproaching 4 mm of the model's surface.

FIG. 2A is a side view of a handheld computer-aided machining tool 201,in a prototype of this invention. FIG. 2B is a cutaway view of the samehandheld machining tool 201, showing some internal parts of the tool.

As shown in FIGS. 2A and 2B, a rotary bit 205 is powered, via a spindle202, by a DC electric motor 221. Three servo motors 209 control theposition of the titanium shaft 235 relative to the handle 203 (and thusthe position of the spindle 202 relative to the handle 203). An MMTSmagnetic sensor 207 is used to track the position of the handheldmachining tool 201 relative to the object being sculpted. Aforce-sensing pressure sensor 215 (FSR) mounted in the handle 203 candetect force applied when the machining tool 201 is pressed against theobject being sculpted. Two LEDs 213, 214 provide visual feedback to theuser regarding, respectively: (i) pressure detected by the FSR sensorand (ii) the distance between the bit 205 and the object being sculpted.Two flexible components 217, 218 are configured to allow the handle 203to move relative to the shaft 235 with at least three degrees offreedom. In this prototype, the flexible components 217, 218 are dustfilters. A PCB 221 housed in the machining tool 201 communicates withthe external main computer and, together with the main computer,controls both shaft movement and spindle speed. Multiple-axislinear/spherical plain bearings 219 allow the shaft 203 to move relativeto the handle 203. Rods (e.g., 231, 232) actuated by the three servomotors 209 are employed to control the position of the shaft 235relative to the handle 203.

FIGS. 3A and 3B show examples of how the multiple-axis bearing 219allows the handle 203 of the handheld machining tool to move in 3degrees of freedom relative to the rest of the tool, including the shaft235. In FIG. 3A, the movement is lateral. In FIG. 3B, the movement is upand down in a carving plane.

In a prototype of this invention, a major part of the computation isdone on a PC (personal computer). An Alienware® M14x Laptop (i7-3740QMIntel® core, 12 GB DDR3, and 2 GB NVIDIA GeForce GT 650M graphic cardare employed. A screen (e.g., 115) provides the user with visualfeedback. An MMTS is used for tracking, because it has no drift and doesnot require an optical line-of-sight. In this prototype, the MMTScomprises a Polhemus FASTRAK® system, which is an AC six DOF system thathas low latency (4 ms), high static accuracy (position 0.76mm/orientation 0.15° RMS), and high refresh rate (120 Hz).

In this prototype, the CAD model resides on the computer. The computerexecutes software that runs in Grasshopper (a plug-in for Rhino). Theinput to the software is the 6D location and orientation of the tool(via serial communication), and the outputs are commands to the controlPCB on the handheld machining tool. The communication with the PCB isone-sided synchronic, using virtual serial communication over Bluetooth.

A prediction of the next position of the bit is extrapolated by a splineof the 4th order (using the current location and the 3 previous ones).The software calculates the distances D to the CAD model (usingrhinoscript MeshCP) from both the current location and the predictedone, and estimates which of these points put the model at higher risk(i.e., which is closer to the model). While the spindle's speed iscalculated on the tool as a factor of Pr and Rs, the latter iscalculated by the main control software (values in mm):

$\begin{matrix}{{Rs} = \left\{ \begin{matrix}0 & {{{{if}\mspace{14mu} D}<={100\mspace{14mu} {and}\mspace{14mu} D} > 4};} \\{D/8} & {{{{if}\mspace{14mu} D}<={4\mspace{14mu} {and}\mspace{14mu} D} > 0};} \\1 & {{elsewhere}.}\end{matrix} \right.} & \left( {{Eq}.\mspace{14mu} 2} \right)\end{matrix}$

For example, in a prototype of this invention, the handheld machiningtool is configured to prevent the user from unintentionally overridingthe CAD model, as follows: (a) if the distance between the highest riskposition of the bit and the model is more than 4 mm, the computercalculates the risk Rs as zero, and does not slow the spindle rotationalspeed Sp; (b) if the distance D between the highest risk position andthe model is greater than 0 mm and less than or equal to 4 mm, thecomputer computes the risk Rs as D/8, and thus slows the spindlerotational speed Sp; and (c) if the highest risk position is within themodel, the computer calculates the risk Rs as 1 and stops the spindlerotation. See Equations 1 and 2. However, if the user has provided inputthat indicates the user's intent to deliberately override the CAD model,then the computer will not protect the 3D model by slowing the spindlein this way.

Equation 2 may be varied, depending on desired characteristics of themachining tool. For example, in some implementations, the spindle willnot rotate if the distance between bit and sculpted object exceeds acertain value (e.g., 10 mm).

FIG. 4 is a high level diagram showing data flow, in a prototype of thisinvention. The MMTS 401 feeds the PC 403 with positioning information.The PC communicates with a control PCB 407, which sends control signalsto the spindle 409 and shaft 411.

FIG. 5 is a high level diagram of control software, in this prototype.The control software is implemented with a Grasshopper plug-in toRhino3D. The computer receives signals from the motion tracking system501 indicative of orientation and position of the handheld machiningtool, and predicts 503 the next position of the bit. Based on thispredicted next position and the CAD model 505, the computer calculates arisk factor 507 to the model, and outputs control signals forcontrolling spindle speed 511. The computer also outputs signals forcontrolling the position of the shaft 509. The shaft control signals 509and spindle rotation speed control signals 511 are sent to the controlPCB 513 on the machining tool. The computer also communicates with agraphical-user interface (GUI) displayed on a screen.

In this prototype, shaft control signals 509 are used to control theposition of the shaft during semi-autonomous machining.

Alternately or in addition, the shaft control signals may be used tocontrol shaft retraction for purposes of preventing the bit fromentering the volume of the 3D model. In this alternate or additionalapproach, the computer prevents the bit from entering the volume of the3D model, causing the servo motors to draw the shaft back as much aspossible. When the bit moves farther from the surface, the softwarepushes the shaft back to its original position.

In exemplary implementations of this invention, shaft control takes intoconsideration the tool's attack angle (α—the angle between the shaft andthe normal to the model's surface in the closest point). The defaultshaft position is fully “open” ahead, with a potential to move 20 mm (ina direction normal to the surface of the 3D model) to absorb the offsetand retract the shaft.

In a prototype implementation of this invention, the user operates thehandheld machining tool while sitting adjacent to the material to besculpted (e.g., balsa foam), which is attached to a wooden stand. Theuser is free to investigate any machining approach, such as extrudinglines, drilling holes, trimming surfaces, or using an arbitrary pattern.The computer slows down the spindle as the bit approaches the model,stopping it completely when it would penetrate the CAD model. Thisenables the user to cut along the boundary of the CAD model wheredesired. The user can also leave parts of the model unfinished oroverride the computer using the pressure sensor. Further, in some modesof operation, the system can dynamically alter the model based on useractions or operate autonomously.

This prototype may operate in at least three interaction modes: (i) astatic mode in which a static CAD model is used, where the computerassists by preventing the user from damaging the model; (ii) a dynamicmode where the computer numerically controls the CAD model, respondingto the user's actions, so that the model dynamically changes duringfabrication; and (iii) an autonomous mode where the computer can operateindependently of the user, for tasks such as semi-automatic texturerendering.

Static Mode:

In the static mode, the user interaction with the tool does notcontribute to the CAD model, and the boundary of the virtual objectremains static during the process. The user determines the tool-path,which not only enables personalization of the produced artifacts, butalso can overcome complicated challenges, such as merging a number of 3Delements into a single hybrid object.

The handheld machining tool provides the user with direct and immediatecontrol over the tool-path for machining (subject, if the protectivefeature is not overridden, to not entering the protected region definedby the CAD model). The surface smoothness and resolution of a machiningtask are determined by the size and shape of the bit and the tool-path.Usually, a manual fabrication process renders a chaotic surface patterncompared to that of an automatic process, which renders an organizednetwork of tool marks (assuming the same cutting bit). Unlike themechanical tool-path, the manual one, which is a product of the maker'sdexterity and patience, never repeats itself, evolving into a singularsurface texture.

FIGS. 6A and 6B show an example of static mode operation. FIG. 6A is avisualization 601 of a static CAD model of a tiger. FIG. 6B is aphotograph of a sculpture 603 of the tiger, which was fabricated using aprototype of the handheld tool and this static model. The final textureis a product of the user tool-path, the properties of the material, thebit size, and the latency of the system. In addition to texturequalities, several parts were left unfinished (the legs), as ademonstration of decisions made during the work. The tool is capable ofachieving a smoother surface than shown in FIG. 6B, as a factor of theuser's patience and finishing time.

During static mode operation, the user can switch between different CADmodels during the work, by changing the reference model in the computerand positioning it virtually. The fusion of these models need not bedetermined numerically as the user inherently resolves these challengesphysically while fabricating, relinquishing the need to solve meshintersection problems and updating a single CAD model to represent theoutput.

For example, FIG. 7 shows a hybrid sculpture 701 that merges asaber-tooth tiger model 713 with dragon wings 709 and deer horns 705,where the parts were virtually aligned and switched among themselvesduring the working process, without having a representative watertightmesh. As shown in FIG. 7, the sculpture has a hybrid 3D shape thatmatches parts of three different meshes (saber tooth tiger 713, deer 703and dragon 707). The hybrid 3D shape does not exist virtually; it existsonly in the fabricated object.

During static mode operation, the user may intentionally override theCAD model, and intentionally remove material from the volume that wouldotherwise be protected by the static model. By overwriting the computerwith the override button on the handle, the user can minimize thedigital control of the spindle. The user still receives feedbackregarding the bit's position with respect to the model (with a sonicalarm and LED). Thus, when the static CAD model is being overridden, itfunctions effectively as a non-binding recommendation. In addition toleaving parts unfinished (as in the earlier example), a user who hasoverrode the protective function can intentionally “damage” the staticmodel, working around or inside the virtual shape. This mode allows forphysical improvisation, without updating the CAD model.

FIG. 8 is a photograph of a sculpture 801, produced by a human useroverriding a static CAD model of a tiger. The result of overriding thecomputer guidance is a completely different model. The artists tookrisks and produced a unique artifact. In the example shown in FIG. 8,the user continued to manually remove parts of the model (e.g., at 803)to achieve a unique artifact, representing real-time manualinterpretations of the CAD model.

Dynamic Mode:

In the dynamic mode, the user can work with a CAD model that dynamicallychanges in response to user actions during the course of fabrication.The dynamic mode allows a user of the handheld tool to work creativelywithin a controlled environment, where a dynamically responsive modelguides the sculpting.

The dynamic mode has at least three submodes: (1) direct shapedeformation, (2) volume occupancy optimization, and (3) data-drivenshape exploration. Each submode is a different way in which a user caninteract with a dynamic CAD model.

A. Direct Shape Deformation (Dynamic Mode)

One type of user interaction in dynamic mode is direct shapedeformation. In this submode, the user can deform the shape of the CADmodel by movements of the handheld machining tool (and other actions)taken during the sculpting. In this submode (direct shape deformation),when the user presses the override button and moves the bit inside theCAD model, the computer deforms the mesh at the risk zone to amelioratethe penetration and loss of material.

In a prototype of this invention, a simple weighted approach is used forlocal deformation with respect to the user's action. As weights for theoffset vector of vertices ({right arrow over (O)}_(v), where v is thevertex index), a Gaussian decay over the geodesic distance from thenearest point to the bit is employed, in order to create an effect of asmooth deformation:

$\begin{matrix}{\overset{\rightarrow}{O_{v}} = {\overset{\rightarrow}{T_{v}}*^{\frac{- {({d_{v}/S})}^{2}}{0.005{({10 - \Pr})}^{2}}}}} & \left( {{Eq}.\mspace{14mu} 3} \right)\end{matrix}$

where Pr{right arrow over (T)} is the penetration vector (the vectorbetween the point of first contact to the deepest bit position), d_(v)is the distance from v to the penetration point, and S is the amount ofaffected vertices, which can be defined by the user (and thus define theaffected area).

FIGS. 9A, 9B, 9C illustrate direct model deformation. In direct modeldeformation, the user can override and deform the CAD model. The usermoves the bit of the handheld machining tool to within the volume of themodel. In response, the computer smoothly deforms the CAD model inproportion to the bit's penetration of the material.

FIG. 9A shows a sculpture 903 of a giraffe produced in accordance withthe original CAD model 901, without a user override. In other words, theuser did not choose to deform the model.

FIGS. 9B and 9C show sculptures 917, 929 of a giraffe, which wereproduced using a CAD model that dynamically deformed in response to theuser's carving actions. In FIG. 9B, the neck of the original CAD model901 became progressively more bent 913, 915 as the bit approached fromthe left. In FIG. 9B, the neck of the original CAD model 901 becameprogressively more bent 923, 925, 927 as the bit approached from theleft and then from the top.

B. Volume Occupancy Optimization (Dynamic Mode)

A second type of user interaction in dynamic mode is Volume OccupancyOptimization (VOO). In VOO, the computer calculates a CAD model thatbest fits the shape of the object sculpted, for example, an irregularpiece of wood. The artist may select the VOO option to try to maximizethe volume of the contained shape as a creative decision, and tominimize the amount of material to remove. The computer-assistedhandheld machining tool allows a user to work in this fashion.

In VOO, the computer can dynamically modify the CAD model in response tothe user's carving actions. As the user changes the shape of thephysical object by removing material, the computer responds bydetermining a new CAD model that maximizes the volume occupancy of thenew physical shape.

In a prototype of this invention, VOO is implemented by the computerusing numerical optimization of volume occupancy for a parametric modelof three parameters.

To illustrate volume occupancy optimization (VOO) according toprinciples of this invention, consider the following example: a bowl,which is a simple parametric object. A plain bowl has three parameters:outer radius (r_(o)), inner radius (r_(i)) and height (c). DenoteΘ={r_(o),r_(i),c}.

In this example, parameteric spheres and cubes can be used to create themodel of the bowl with constructive solid geometry (CSG) booleanoperations (implemented by the Carve CSG library).

In order to fit the bowl shape in the material, the remaining volume isdetermined. To do so, as the handheld machining tool carves (withoutrestriction) a part of a rectangular piece of material, position data isgathered by the MMTS and used to calculate the toolpath. The computercalculates the remaining volume of the material by applying thresholdson the x, y and z positions of the bit to determine what was removed.

Insofar as each discrete point along the toolpath describes only thecenter of the bit, rather than the complete volume the bit had removed,multiple points (e.g., 10 points) are sampled on a sphere with radiusequal to the actual bit size (e.g., 3.2 mm) to represent the entirevolume of the bit as it passed through space. This point-cloud describesthe volume where the bit removed material.

In this example, a solid shape is created out of the tool-path pointcloud using the Alpha Shapes method. Once the removed portion iscomputed, the remaining volume is easily determined.

In this example, given the remaining volume, the computer calculates aparametric bowl that maximizes a multivariate score function is fittedinside. The score function is as follows:

f ₁(Θ)=w ₁ *V _(remain)(Θ)

f ₂(Θ)=w ₂ *V _(out)(Θ)

f ₃(Θ)=w ₃*(1−c)

f ₄(Θ)=w ₄*(1−r _(in))

F(Θ)=[f ₁(Θ);f ₂(Θ);f ₃(Θ);f ₄(Θ)]  (Eq. 4)

The score function returns a vector. The computer minimizes the norm ofthe vector. The V_(remain)(Θ) marks the remaining volume and V_(out)(Θ)marks the volume that the shape takes outside the remaining volume, i.e.out in the air. Both these measures are minimized in order to maximizeoccupancy and minimize escape. The bowl is also made to be as high andthick as possible, which is embodied in the final two residuals.

In this example, a non-linear least-squares solver is used to find thesolution for the following optimization problem:

arg min_(Θ) ∥F(Θ)F∥ ²  (Eq. 5)

In this example, because of the CSG operations, the function isevaluated numerically. To bootstrap the parametric search a grid searchof the space is first used to find a starting point; the solver thenfinds the optimal solution and thus the best fitting bowl to the givenvolume.

C. Data-Driven Shape Exploration (Dynamic Mode)

A third type of user interaction in dynamic mode is Data-Driven ShapeExploration (DDSE). In DDSE, the computer-assisted handheld machiningtool uses a database of shapes to dynamically modify the CAD model, inresponse to carving motions made by the user.

For example, consider a humanoid model, which is a canonical pursuit insculpting and drawing.

In order to construct the database, in a prototype of this invention,over 4000 examples of human pose are recorded with the Kinect sensor viathe OpenNI software stack. The poses are clustered using a K-Meansvariant, to 50 clusters (meta-poses) of varying sizes, using WEKA(Waikato Environment for Knowledge Analysis). Then the humanoid model isauto rigged to a skeleton model that corresponds with the Kinect. Fordeformation of the mesh the Linear Blend Skinning method was used.

The process of finding the remaining volume, employed in VOO, is usedhere. Then, a search over the clustered database is performed to findthe pose that has the least amount of escape from the remaining volume(V_(out)), followed by a search within the best found cluster. In eachiteration, the user chooses from multiple options for advancement. Afterthe database search is concluded, fine tuning ensues, for the positionof the limbs (arms and legs) and for small translations (on the groundplane) of the entire shape in respect to the volume. A grid-search ofthe parametric space is performed to find the best tuning of theselected pose.

Autonomous Operation Mode:

In exemplary implementations of this invention, independent actuation ofthe shaft operates semi-autonomously: while the user holds the handheldmachining tool and makes large-scale movements, the tool makesautonomous smaller-scale movements. Thus, the tool can be operated as asemi-autonomous machining tool at small scale.

FIG. 10 is a photograph showing regions milled as the bit moves in astraight line through material. In some regions, the semi-autonomousmode was not used, creating a narrow cut. In other regions, the tooloperated semi-autonomously to remove more material, resulting in alarger virtual bit.

The semi-autonomous mode can be used for texture rendering. For example,this mode can be used to render a fur texture. When the bit is closerthan 4 mm to the fur segment, the servos operate with a linear peckingmovement (4 Hz, 5 mm movement range) to achieve a fur texture on theteddy bear surface. The user continues to operate the tool freely, notconstrained by the shaft actuation.

Surface textures can be created using autonomous mode operation. Forexample, a CAD model of a teddy bear can be embellished with furtextures. The mesh surface can be encoded with either a rough texture orsmooth texture. Rough texture causes the shaft to move back and forth,creating dimples in the material that simulate fur.

A user may input selections to change the interaction mode (e.g.,static, dynamic DSD, dynamic VOO, dynamic DDSE, or autonomous).

Examples of Use of Prototype

Here are some non-limiting examples of use of a prototype of thisinvention. The prototype was used to fabricate multiple completeartifacts. The tool carved both high and low density balsa foams,basswood, and carving wax. The control software updated at a frame ratevarying between 10 to 20 frames per second (FPS). The computer workedwith mesh models of 150 vertices (humanoid) to 5370 vertices (gargoyle),lengths between 120 mm (giraffe) to 280 mm (gargoyle), and withproduction times of 40 minutes (giraffe) to 5 hours (gargoyle). Thesystem static-bit accuracy (measured by holding the bit in one placewhile rotating the tool around it) varied between 0.05 mm RMS (20 cmfrom the magnetic field generator) to 0.4 mm RMS (70 cm away).

The surface accuracy of the tool (i.e., how accurately the tool canreconstruct a predesigned CAD model) depends on, among other things, theframe rate, tool movement speed, and material density. For example, inone case, with 15 FPS and 350 mm/sec attack speed, the bit penetrated3.5 mm in a dense balsa foam before the system shut down the spindlerotation. However, adherence of the resulting surface to the CAD modelalso depends on the maker's dexterity and patience when working with thetool.

In illustrative uses of the prototype, the user often started by cuttingmaterial farther from the objects, usually by guiding the tool alonghorizontal and vertical lines, removing material from one side toanother. Once the model became recognizable to the human, the operationoften changed to tracking the surface manifolds with the tool. Thesemachining methods were typically intuitively decided by the user duringthe work, rather than being planned in advance. At times when there wasa risk to harm the model the tool prevented a mistake by stopping thespindle, applying an invisible “protection field” around the object. Thechanges in the spindle speed, when the bit approaches the model surface,informed the user regarding the relative location of the bit withrespect to the model. This feedback helped the user identify theinvisible “protection field” the system creates around the shape insidethe material.

In illustrative uses of the prototype, the display screen presents avisualization of the CAD model. In this visualization, a mark displayedon the screen represents the current position of the tool's bit.Occasionally, the user relied on this mark during the work, especiallyin the initial stage where the virtual shape is not yet revealed in theraw material.

This invention may be implemented with a projector for projecting visualfeedback of the CAD model on the material being sculpted.

DEFINITIONS AND CLARIFICATIONS

Here are a few definitions and clarifications. As used herein:

The terms “a” and “an”, when modifying a noun, do not imply that onlyone of the noun exists.

The term “bit” shall be construed broadly. The term “bit” includes (i)any milling cutter and (ii) any other tool configured to remove materialfrom an object by shear deformation or abrasion while in physicalcontact with the material. A “bit” may, for example, operate by eitherlinear or rotary motion.

The term “CAD model” shall be construed broadly. For example, the term“CAD model” includes any model that (i) was created or modified, atleast in part, by a computer and (ii) is stored in electronic memory. ACAD model may be of any object. For purposes of this definition of CADmodel: (i) the term “model” shall be construed broadly and shall includeany model, design, mesh or representation, and shall further include anyset of data representing points in a 3D volume or positions in orelements of a 3D object, including the surface or interior of a 3Dobject, and (ii) the term “object” shall be construed broadly to includeany shape, object or structure. A CAD model does not necessarily controlthe shape of an object produced by using the CAD model. For example, insome cases, a CAD model may function effectively as a non-bindingrecommendation or suggestion.

The term “carve” (and similar terms, such as “carving”) shall beconstrued broadly. For example, carving includes any method of removingmaterial from an object.

The term “cutting speed” shall be construed broadly. The term “cuttingspeed” may apply to either a rotary or linear cutting tool. In eachcase, the “cutting speed” is the speed difference between the cuttingtool and the workpiece. For example, in the context of a rotary cuttingtool, “cutting speed” is the linear tangential equivalent (at thetool/workpiece interface) of the spindle speed. The term “cutting speed”does not include feed speed.

The term “comprise” (and grammatical variations thereof) shall beconstrued broadly, as if followed by “without limitation”. If Acomprises B, then A includes B and may include other things.

The term “e.g.” means for example.

The fact that an “example” or multiple examples of something are givendoes not imply that they are the only instances of that thing. Anexample (or a group of examples) is merely a non-exhaustive andnon-limiting illustration.

Unless the context clearly indicates otherwise: (1) a phrase thatincludes “a first” thing and “a second” thing does not imply an order ofthe two things (or that there are only two of the things); and (2) sucha phrase is simply a way of identifying the two things, respectively, sothat they each can be referred to later with specificity (e.g., byreferring to “the first” thing and “the second” thing later). Forexample, unless the context clearly indicates otherwise, if an equationhas a first term and a second term, then the equation may (or may not)have more than two terms, and the first term may occur before or afterthe second term in the equation. A phrase that includes “a third” thing,a “fourth” thing and so on shall be construed in like manner.

The terms “horizontal” and “vertical” shall be construed broadly. Forexample, “horizontal” and “vertical” may refer to two arbitrarily chosencoordinate axes in a Euclidian two dimensional space.

The term “include” (and grammatical variations thereof) shall beconstrued broadly, as if followed by “without limitation”.

The term “or” is inclusive, not exclusive. For example “A or B” is trueif A is true, or B is true, or both A or B are true. Also, for example,a calculation of “A or B” means a calculation of A, or a calculation ofB, or a calculation of A and B.

A parenthesis is simply to make text easier to read, by indicating agrouping of words. A parenthesis does not mean that the parentheticalmaterial is optional or can be ignored.

The term “machining” shall be construed broadly. For example, machiningincludes (i) any removal of material by shear deformation or abrasion,(ii) any removal of matter from a first object caused by mechanicalmovement of one or more other objects relative to the first object, or(iii) any other removal of material from a solid object. “Machining”includes, for example, cutting, carving, milling, lathing, drilling,boring, broaching, sawing, shaping, planing, turning, reaming, tapping,grinding, and abrading. Machining may occur, for example, by eitherrotary motion or linear motion of a part relative to the object beingmachined. “Machining” also includes all other methods of subtractivefabrication, including electrical discharge machining, electrochemicalmachining, electrochemical machining, electron beam machining,photochemical machining, ultrasonic machining, laser ablation, and lasercutting. A “machining tool” includes, for example (i) any tool that cutsmaterial with a rotary bit, (ii) any milling tool, or (iii) any toolthat removes material from another solid object. Also, for example, a“machining tool” includes any mill, lathe, drill, saw, blade, scraper,awl, broach, laser cutter, or any tool with a single cutting point ormultiple cutting points, including at any scale. Also, for example, a“machining tool” includes any tool configured for machining. A slowingor halt of machining may comprise, for example, a reduction or halt ofcutting speed, in the case of a machining tool that operates by a linearor rotary cutting motion. The term “machining” does not imply that theremoval of material is controlled by a machine or by a static design.

The term “sculpture” (and similar terms, such as “sculpt”) shall beconstrued broadly. For example, sculpting includes any method ofremoving material from an object.

The term “tool” shall be construed broadly. The term “tool” includes (i)any physical item that can be used to achieve a goal, and (ii) anyinstrument, utensil, implement, machine, apparatus or equipment.

Variations:

This invention may be implemented in many different ways, and is notlimited to the embodiments described above. Here are some non-limitingexamples.

This invention may, for example, comprise (i) a milling tool, (ii) alinear or rotary cutting tool, or (iii) a machining tool.

This invention may, for example, be implemented using any computerassisted handheld apparatus configured for subtractive fabrication oradditive fabrication.

This invention may be implemented as apparatus comprising a machiningtool, wherein the machining tool: (a) is configured to be handheld by ahuman user while effecting removal of material from an object; (b)includes a bit; (c) is configured to be able to perform a function, thefunction comprising slowing or halting the removal of material dependingon position of the bit relative to a physical region defined by a CADmodel; (d) is configured to enable or disable the function, in responseto input from the user, and (e) is configured to accept in real time orto compute in real time modifications to the CAD model, whichmodifications to the CAD model are determined, at least in part, by atleast some movements of the bit relative to the object during themachining. Furthermore: (1) the modifications to the CAD model maycomprise deformation of a surface of the CAD model, which deformationminimizes either (i) penetration of the bit into the physical regiondefined by the CAD model, (ii) amount of material removed from thephysical region defined by the CAD model, or (iii) another value thatdepends at least in part on the penetration or the amount of materialremoved; (2) the modifications may alter the CAD model to maximizepercentage, by volume, of the object that is occupied by the physicalregion defined by the CAD model; (3) the modifications may be computed,at least in part, by selecting, out of a set of multiple CAD models, aselected CAD model that maximizes percentage, by volume, of the objectthat is occupied by the physical region defined by the selected CADmodel; (4) the apparatus may be further configured to actuate the bit tomove in paths relative to the object that are not determined by theuser; (5) the apparatus may include at least one sensor for detectingdata indicative of position of the machining tool relative to theobject, (6) the apparatus may include at least one sensor for detectingdata indicative of pressure exerted by the machining tool on the object;(7) the apparatus may include one or more transducers for providingfeedback to the user regarding at least one of a group consisting of:(i) position of the machining tool relative to the object, or (ii)pressure exerted by the machining tool on the object; (8) themodifications to the CAD model may be computed, at least in part, by oneor more computers that are external to the machining tool; (9) theapparatus may further comprise the one or more computers; (10) differentCAD models may be used for different portions of the object, inaccordance with selections inputted by the user; and (11) the functionmay protect a hybrid physical region, which region does not correspondto any single CAD model.

This invention may be implemented as a method comprising using amachining tool, wherein the machining tool: (a) is configured to behandheld by a human user while effecting removal of material from anobject; (b) includes a bit; (c) is configured to be able to perform afunction, the function comprising slowing or halting the removal ofmaterial depending on position of the bit relative to a physical regiondefined by a CAD model; (d) is configured to enable or disable thefunction, in response to input from the user, and (e) is configured toaccept in real time or to compute in real time modifications to the CADmodel, which modifications to the CAD model are determined, at least inpart, by at least some movements of the bit relative to the objectduring the machining. Furthermore: (1), the modifications to the CADmodel may comprise deformation of a surface of the CAD model, whichdeformation minimizes either (i) penetration of the bit into thephysical region defined by the CAD model, (ii) amount of materialremoved from the physical region defined by the CAD model, or (iii)another value that depends at least in part on the penetration or theamount of material removed; (2) the modifications may alter the CADmodel to maximize percentage, by volume, of the object that is occupiedby the physical region defined by the CAD model; (3) the modificationsmay be computed, at least in part, by selecting, out of a set ofmultiple CAD models, a selected CAD model that maximizes percentage, byvolume, of the object that is occupied by the physical region defined bythe selected CAD model; (4) the apparatus may be further configured toactuate the bit to move in paths relative to the object that are notdetermined by the user; and (5) different CAD models may be used fordifferent portions of the object, in accordance with selections inputtedby the user.

This invention may be implemented as apparatus comprising a machiningtool, wherein the machining tool: (a) is configured to effect removal ofmaterial from an object, the removal of material having an operativerate and occurring within an operative zone; (b) is configured to behandheld by a human user during the removal of material; (c) isconfigured to be able to perform a function, the function comprisingslowing or halting the operative rate depending on position of theoperative zone relative to a physical region defined by a CAD model; (d)is configured to enable or disable the function, in response to inputfrom the user, and (e) is configured to accept in real time or tocompute in real time modifications to the CAD model, which modificationsto the CAD model are determined, at least in part, by at least somemovements of the operative zone relative to the object during theremoval of material.

This invention may be implemented as apparatus comprising a tool forsubtractive fabrication or additive fabrication, wherein the tool: (a)is configured to effect removal of material from or deposition ofmaterial to an object; which removal or deposition has an operative rateand occurs within an operative zone; (b) is configured to be handheld bya human user during the removal or deposition; (c) is configured to beable to perform a function, the function comprising altering theoperative rate depending on position of the operative zone relative to aphysical region defined by a CAD model; (d) is configured to enable ordisable the function, in response to input from the user, and (e) isconfigured to accept in real time or to compute in real timemodifications to the CAD model, which modifications to the CAD model aredetermined, at least in part, by at least some movements of theoperative zone relative to the object during the removal or depositionof material.

CONCLUSION

It is to be understood that the methods and apparatus that are describedherein are merely illustrative applications of the principles of theinvention. Numerous modifications may be made by those skilled in theart without departing from the scope of the invention.

What is claimed is:
 1. Apparatus comprising a machining tool and atracking system, wherein: (a) the tracking system is configured to trackposition of a bit of the machining tool relative to an object; and (b)the machining tool: (i) includes a shaft that is attached to the bit;(ii) is configured to be held by a human user while effecting physicalremoval of material from the object, such that the weight of the tool issupported solely by a hand of the user during the removal, (iii) isconfigured to be controlled depending on the tracked position of the bitrelative to the object, to prevent the bit from entering a protectedregion, by causing the shaft to undergo retraction, such that during theretraction the shaft moves relative to a handle of the machining tool;and (iv) is configured to calculate, in response to a movement of thebit relative to the object during the physical removal of material, achange in the boundary of the protected region; wherein the protectedregion is defined by a computer-aided design (CAD) model.
 2. Theapparatus of claim 1, wherein the bit moves away from the protectedregion during the retraction.
 3. The apparatus of claim 2, wherein themachining tool is configured such that, after the retraction, the shaftis returned to an original position.
 4. The apparatus of claim 1,wherein the shaft is moveable, in multiple degrees of freedom, relativeto a handle of the apparatus.
 5. The apparatus of claim 4, wherein theshaft is moveable laterally, relative to the handle.
 6. The apparatus ofclaim 1, wherein the apparatus includes multiple-axis bearingsconfigured such that the shaft is moveable, in multiple degrees offreedom, relative to a handle of the apparatus.
 7. The apparatus ofclaim 1, wherein the apparatus includes a button for accepting inputthat comprises an instruction that overrides computer control that wouldotherwise prevent the bit from entering the protected region.
 8. Theapparatus of claim 1, wherein: (a) the apparatus includes a pressuresensor for measuring pressure; (b) the bit is a rotary bit and themachining tool includes a motor for powering the rotary bit; and (c) themotor is configured to have a motor speed that is a linear factor ofparameters that include the pressure.
 9. The apparatus of claim 1,wherein the change in a boundary of the protected region minimizes thevolume of material that needs to be removed from the object to achieve atarget shape.
 10. The apparatus of claim 1, wherein the change in theboundary of the protected region is computed by selecting, out of a setof virtual models of protected regions, a model that minimizes thevolume of material that needs to be removed from the object to achieve atarget shape.
 11. A method comprising: (a) a tracking system trackingposition of a bit of a machining tool relative to an object; (b) amachining tool physically removing material from an object; (c)controlling the machining tool, depending on the tracked position of thebit relative to the object, to prevent the bit from entering a protectedregion, by retracting a shaft that is attached to the bit, such thatduring the retracting the shaft moves relative to a handle of themachining tool; and (d) calculating, in response to a movement of thebit relative to the object during the physical removal of material, achange in the boundary of the protected region; wherein (i) the tool isconfigured to be supported solely by a hand of a user during the removalof material, and (ii) the protected region is defined by acomputer-aided design (CAD) model.
 12. The method of claim 11, wherein,the bit moves away from the protected region during the retracting. 13.The method of claim 12, wherein the machining tool is configured suchthat, after the retracting, the shaft is returned to an originalposition.
 14. The method of claim 11, wherein the shaft is moveable, inmultiple degrees of freedom, relative to a handle of the apparatus. 15.The method of claim 11, wherein the shaft is moveable laterally,relative to the handle.
 16. The method of claim 11, wherein themachining tool includes multiple-axis bearings configured such that theshaft is moveable, in multiple degrees of freedom, relative to a handleof the apparatus.
 17. The method of claim 11, wherein the machining toolincludes a button that accepts input, which input overrides computercontrol that would otherwise prevent the bit from entering the protectedregion.
 18. The method of claim 11, wherein: (a) a pressure sensor inthe machining tool measures pressure; (b) the bit is a rotary bit; (c) amotor in the machining tool powers the rotary bit; and (c) the motor hasa motor speed that is a linear factor of parameters that include thepressure.
 19. The method of claim 11, wherein the change in a boundaryof the protected region minimizes the volume of material that needs tobe removed from the object to achieve a target shape.
 20. The method ofclaim 11, wherein the change in the boundary of the protected region iscomputed by selecting, out of a set of virtual models of protectedregions, a model that minimizes the volume of material that needs to beremoved from the object to achieve a target shape.